A study on objective structured practical examination ( OSPE ) as a tool for assessment of medical students

Background: Most of the medical colleges in India follow the Conventional Practical Examination (CPE) system for evaluation of practical or clinical skills. This method has several flaws such as inter-examiner marks variation, favouritism, varying difficulty level of different experiments etc. which prevents uniform marking. Objective Structured Practical Examination (OSPE), conceptualised by Harden about 50 years ago reduces these flaws. Hence this study was carried out to compare the marks of the student in CPE and OSPE followed by a student’s feedback on OSPE. Methods: Randomized, prospective educational trial was carried out on 100 second year MBBS students. Students were subjected to CPE followed by OSPE. They were evaluated by four examiners. Marks were expressed as percentage. A Likert’s scale based questionnaire on OSPE was prepared and distributed among the students. Their views and perception towards OSPE were noted. The results were analysed by SPSS version 17.0. P value <0.05 was considered significant. Results: Average mark scored by the students in OSPE (73.0±9.8 %) was higher as compared to CPE (70.7±9.6 %) which was statistically significant. There was no inter-examiner variation observed in OSPE. Above 90% of students felt that OSPE was more practically oriented examination with less chances of bias. More than 75% felt that OSPE should be used for practical examination. Conclusion: OSPE is better tool for assessment as the inter-examiner variation and bias is eliminated. It is therefore strongly recommended to implement OSPE as an assessment tool in all Indian medical colleges. Introduction An integral part of a medical curriculum is an appropriate assessment of the students’ clinical/practical competencies. Assessment drives learning. But to foster active learning, assessment needs to be informative. University examination in medical college all over India mainly consists of theory and practical examinations. While the theory examination is mainly concerned with the assessment of knowledge, the purpose of practical or clinical examination is to assess the cognitive, psychomotor andaffective domain of the student. However, this purpose is not met with Conventional Practical Examination (CPE) which is presently being carried out in most of the medical colleges in India. The CPE is highly subjective which raises concerns over its validity and reliability.Moreover, CPE mainly focuses on the “knows” and “knows how” aspects, i.e. the base of the ‘Miller’s pyramid of competence’. Another limitation with CPE is that the students are not assessed continuously and the marks are awarded based on the questions answered by Indian Journal of Basic and Applied Medical Research; March 2016: Vol.-5, Issue2, P. 784-790 785 www.ijbamr.com P ISSN: 2250-284X , E ISSN : 2250-2858 them at the end of the practical. Therefore, the students are tested for their knowledge alone and not for attitude and skills.It is well known that “learning is assessment based”,therefore, the students learn for what they are assessed.Therefore there is a need to form a mode of assessment which would be less subjective and more objective. Hence, Objective Structured Practical Examination (OSPE) and Objective Structured Clinical Examination (OSCE) in Basic Medical Sciences and Clinical Sciences respectively, have being introduced, to overcome the problems which are faced with CPE in medical education. OSPE is a practical exam system which is not just objective but reliable and valid as well. It consists of a series of stations at which students work through tasks designed to test various skills. The responses of the students are evaluated using agreed checklists. It also eliminates examiner bias. Aim of the study: This study was hence taken up with the aim to compare the scoring pattern of students by CPE andOSPE. The examination process was then followed by feedback from the students regarding their perception towards OSPE as a new tool of assessment. Material & methods This study was a randomized, prospective educational trial carried out by the Department of Pharmacology on 100 second year MBBS students of Rohilkhand Medical College, Bareilly. Institutional ethical committee clearance and informed consent from all the students and examiners were taken before the study. The syllabus for the test was announced to the students 15 days in advance to ensure they had enough time to prepare for the test. They were subjected to CPE and OSPE on the same topic. For CPE the students were divided randomly into four batches of 25 each. Each group of student was evaluated by a different examiner. Following this all the students were subjected to OSPE. All the examiners and students were introduced to the concept of OSPE. OSPE question were constructed from the same syllabus as CPE. The questions were subjected to peer-review and ambiguous questions were deleted. The process of examination and marking scheme were explained in detail. Any queries related OSPE were cleared. The OSPE consisted of five stations. Four stations had questions based on different domains of learning (Table 1). The fifth station was a rest station. Every station had duration of three minutes. Check list for each station was prepared. An examiner (silent observer) was present on the station with a question on psychomotor domain. The four examiners evaluated the OSPE response sheets of all the students with the checklist provided to them. The score of the tests was expressed as percentage. A questionnaire on OSPE was prepared and distributed among the students after the test (Table 2). Their views and perception regarding OSPE were noted. The questionnaire consisted of five questions based on Likert’s scale. The marks obtained in both the forms of assessment and the response of the questionnaire was analysed statistically which SPSS version 17.0. Independent ‘t’ test was used to compare the mean marks scored in CPE and OSPE. ANOVA was used to compare the marks given by different examiner. P value less than 0.05 was considered significant. Indian Journal of Basic and Applied Medical Research; March 2016: Vol.-5, Issue2, P. 784-790 785 www.ijbamr.com P ISSN: 2250-284X , E ISSN : 2250-2858 Table 1: Station details and domains assessed in OSPE S. No Items of the questionnaire 1 Is OSPE a fair method of examination as compared to CPE? 2 Is OSPE more practically oriented as compared to CPE? 3 Is it easier to score better in OSPE as compared to CPE? 4 Does OSPE reduce the chances of bias by examiners as compared to CPE? 5 Should OSPE be included as assessment tool for evaluation? Table 2: Feedback questionnaire on OSPE Results The response rate of the students was 100% as all the students appeared for both the tests. In CPE the mean marks scored by the student was 70.7 ± 9.6 % as compared to 73.0 ± 9.8 % scored in OSPE (Table 3). Marks given by the four different examiners are shown in Table 4. This difference of mean scores between the two tests was statistically significant (p<0.05). Based on the response to the questionnaire, students’ perception towards OSPE was analysed (Fig 1). 90% of the students found OSPE to be fairer in comparison to CPE. The rest 10% were not sure if OSPE was better than CPE. 94% students felt that OSPE was more practically oriented than CPE and 92% opined that OSPE decreases the bias involved in the assessment process. In terms of ability to score, 63% of said that scoring better marks in OSPE was easier, where as 19% felt that better marks would be obtained in CPE. When asked whether OSPE should be included as a preferred tool for assessment over CPE, 75% agreed and 7% disagreed. The rest 18% were not able to decide between OSPE and CPE. Station No. Station Details Domains Assessed 1 Identify instrument Mention its use Cognitive domain (knowledge) 2 Clinical problem given To write diagnosis & treatment Application Analysis Synthesis 3 Rabbit handling for instilling drugs in its eye Psychomotor Affective 4 To prepare different mixtures/lotions & dispense it Comprehension Application 5 Blank/Rest station Nil